Papers with sign recognition
WLASL-LEX: a Dataset for Recognising Phonological Properties in American Sign Language (2022.acl-short)
Copied to clipboard
| Challenge: | Signed Language Processing (SLP) is a major form of NLP, but has been overlooked by the NLP community. |
| Approach: | They leverage existing resources to construct a large-scale dataset of American Sign Language signs annotated with six different phonological properties. |
| Outcome: | The proposed model outperforms existing approaches on signs unobserved during training. |
Linguistically-driven Framework for Computationally Efficient and Scalable Sign Recognition (L18-1)
Copied to clipboard
| Challenge: | a new general framework for sign recognition from monocular video is presented . the framework exploits state-of-the-art learning methods while incorporating features based on what we know about the linguistic composition of lexical signs. |
| Approach: | They propose a general framework for sign recognition from monocular video . they exploit state-of-the-art learning methods while incorporating features from linguistic information . |
| Outcome: | The proposed framework exploits state-of-the-art learning methods while incorporating features based on what we know about linguistic composition of lexical signs. |
The American Sign Language Knowledge Graph: Infusing ASL Models with Linguistic Knowledge (2025.findings-naacl)
Copied to clipboard
| Challenge: | Sign language models could make language technologies more accessible to deaf and hard-of-hearing signers, but the supply of accurately labeled data struggles to meet the demand associated with training large, end-to-end architectures. |
| Approach: | They construct an American Sign Language Knowledge Graph from 11 sources of linguistic knowledge and use it to train neuro-symbolic models on ASL video input tasks. |
| Outcome: | The proposed model achieves 91% accuracies for isolated sign recognition, 14% for predicting the semantic features of unseen signs, and 36% for classifying the topic of Youtube-ASL videos. |
An HMM Approach with Inherent Model Selection for Sign Language and Gesture Recognition (2020.lrec-1)
Copied to clipboard
| Challenge: | despite the extensive use of HMMs for sign recognition, determining the HMM structure remains a challenge . despite their success in modeling sequential and multivariate data, establishing the structure remains challenging . |
| Approach: | They propose a continuous HMM framework for modeling and recognizing isolated signs . they propose to optimize the number of states for each sign separately during recognition . |
| Outcome: | The proposed model performs better on three different datasets and is competitive with existing models. |
Linguistically Motivated Sign Language Segmentation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Sign language segmentation is a crucial task in sign language processing systems. |
| Approach: | They propose to combine two kinds of segmentation: segmentation into individual signs and segmentation to segment into phrases, larger units comprising several signs. |
| Outcome: | The proposed model is based on linguistic cues observed in sign language corpora and replaces the predominant IO tagging scheme with BIO taging to account for continuous signing. |